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(54) User Interface high-lighter function to provide directed Input for image processing 



(57) A user interface high-lighter function for scan- 
ner software that allows a user to automatically select a 
portion or several disjoint portions of text within a region 
of text in a preview scan of a document When activated, 
the high-lighter function causes the normal mouse cur- 
sor to be replaced with a high-lighter icon (214) having 
an active edge (244). Using the mouse, the active edge 
(244) of the high-lighter icon (214) is moved across a 
desired portion of text, and background pixels traversed 
are displayed in a different color. The traversed pixels 
are then used as input for an image processing routine. 
Text regions are generated from the traversed pixels. 
The beginning and ending of lines not high-lighted are 
cropped. A polygonal bounding box (416, 418, 420) is 
then displayed around the selected text An un-htghJight 
mode can be activated to reduce the selected text An 
optimized re-scan of the selected text can now be per- 
formed. 
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Description 

CROSS REFERENCE TO RELATED APPLICATIONS 

[0001] This application is a cortorujation-in-part of 
Application Serial No. 09/164.693 of Jeffrey P. Lee et al . 
filed October 1. 1998 entitled Click and Select User 
Interlace for Document Scanning. 

TECHNICAL FIELD 

[0002] This invention relates to document scanners 
and more particularly to a user interface for document 
scanners. Even more particularly, the invention relates 
to a user interface high-lighter function for a user inter- 
face for document scanners that allows a user to auto- 
matically select a portion of text, or several disjoint 
portions of text, within one or more regions of text in a 
preview scan of a document or documents for directed 
input for image processing. 

BACKGROUND OF THE INVENTION 

[0003] Current scanner technology allows a user to 
capture and utilize all or portions of various scanned 
documents, images, objects, etc. for use within various 
computer applications, even when the documents are 
comprised of a variety of different components. A docu- 
ment or documents containing text, black and white 
and/or color phottx^aphs, graphics, and color and black 
and white line art can be scanned in their entirety. Alter- 
natively, the user may choose to select only certain por- 
tions of the original document or documents for 
scanning by utilizing scanner software to select an area 
or particular image contained in a preview scan of the 
document or documents. The selected area is then opti- 
mally re-scanned to produce a final image that can be 
utilized in an end application. 
[0004] The Click and Select User Interface for Docu- 
ment Scanning allows the user to automatically select a 
region of interest from the preview scan of the document 
or documents by positioning a mouse cursor within the 
region of interest and then clicking the mouse button. 
Image processing is performed by the scanner software 
starting with the click point as input and a bounding box 
is then displayed around the region of interest as deter- 
mined by the image processing. The user then has the 
ability to readily override the first estimate of the bound- 
ing box, if it proves not to be what the user wanted, by 
using other functions of the user interface. 
[0005] However, there are some situations where, in 
utilizing the Click and Select User Interface for Docu- 
ment Scanning, the user would not be able to override 
the first est ima te of the bounding box to select only the 
desired region of interest to send to an end application. 
Examples are when a user wishes to select only a por- 
tion of a text region that cannot be confined in a rectan- 
gular box, or the user wishes to select two or more 



disjoint potions of text within a text region. 
[0006] It is thus apparent that there is a need in the art 
for an improved method or apparatus which solves the 
objects of the invention. The present invention meets 

5 these and other needs in the art. 

[0007] This application is a continuation-in-part of 

Application Serial No. of Jeffrey 

P. Lee et al . filed entitled Click and Select 

User Interface for Document Scanning, which is incor- 

w porated herein by reference for an that is disclosed and 
taught therein. 

SUMMARY OF THE INVENTION 

is [0008] It is an aspect of the present invention to pro- 
vide a user interface for scanner software that aDows a 
user to easily select specific portions of text regions 
from a preview scan of a document or documents. 
[0009] It is another aspect of the invention to use a 
20 high-fighter function, which is an easy, logical, and famil- 
iar motif, for selecting specific portions of text regions 
from a preview scan of a document or documents. 
[0010] Yet another aspect of the invention is to allow a 
user to use a high-lighter function to select two or more 
25 disjoint potions of text within text regons from a preview 
scan of a document or documents. 
[001 1 ] Still another aspect of the invention is to utilize 
the pixels traversed by an active edge of a high-lighter 
icon as input to an image processing routine to deter- 
so mine a selected bitmapped data area. 

[0012] A further aspect of the invention is to display a 
polygonal bounding box around the selected bitmapped 
data area as determined by an image processing rou- 
tine. 

35 [0013] A still further aspect of the invention is to dese- 
lect portions of text that have been selected utilizing a 
high-lighter icon in an un-high-lighter mode. 
[0014] Another aspect of the invention is to utilize the 
. selected bitmapped data area represented by the 

40 selected text as input for an optimized re-scan of the 
document or documents. 1 

[0015] The above and other aspects of the invention 
are accornplished in a user interface for scanner soft- 
ware that allows a variable resolution preview scan of a 

45 document or documents to be presented to the user in 
a variable resolution preview window of a computer 
monitor. In the preferred embodiment of the invention, 
the Cfick and Select User Interface for Document Scan- 
ning is used allowing a user to click on a point within a 

so region of interest in the variable resolution preview win- 
dow that represents the underlying bitmapped data from 
the variable resolution preview scan. Based on the char- 
acteristics of the bitmapped data represented at the 
click point an image processing routine determines an 

55 area that encompasses the click point and the neighbor- 
ing pants that rtave similar characterise 
bitmapped data at the click point This area is then clas- 
sified by type based on the characteristics of the bit- 
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mapped data represented withi n the area, such as text, 
grayscale image, color image, or black and white image. 
A selection marker is then automatically displayed 
around the area as a first estimate of what the user 
intended by making the mouse click at the click point 
[0016] The selection marker may be a bounding box 
rectangular in shape. For non-rectangular "lasso" 
regions, image analysis software may be applied, auto- 
matically tracing around the lasso region of interest 
Portions of the scanned document or documents that lie 
outside the selection marker are grayed out Based on 
the determination of the type of area, adjustments are 
made automatically to data type, exposure, color, reso- 
lution, and sharpness settings. 
[0017] H the selection marker automatically generated 
does not represent the image of interest desired by the 
user, the user can adjust the size of the region of inter- 
est to include more area or include less area in several 
different ways. One such way is by dragging the selec- 
tion marker handles appropriately, expanding or con- 
tracting the selected area. Or, the user may 
simultaneously click on a point and hold down a control 
key on a keyboard, or right click on the mouse, to either 
expand the region or image of interest to include addi- 
tional area, or contract the region or image of interest to 
exclude area already selected. The user may also right 
di*onthenrxxjsetop<X5upa(xxTtextrrmriu, ardselert 
an "Expand Selection" option or select a "Contract 
Selection" option from the menu. In addition, a different 
region of interest can be selected by the user by clicking 
in an unselected area of the variable resolution preview 
scan, and dragging the mouse to create a rectangular 
area bounded by a rectangular selection marker. 
[0018] Based on the type of image, floating windows 
displayed on the monitor having tools for adjusting vari- 
ous aspects of the selected area are automatically ena- 
bled or disabled from user input Once the region of 
interest is property selected, and any adjustments have 
been made, the user can use a mouse to drag and drop 
the selected area onto the desktop, a writeable folder, or 
onto an open application to launch an optimized final 
scan, or re-scan, of the selected area. After the opti- 
mized re-scan, the resulting image data resides in the 
desktop folder, in a file, or within the open application. 
Alternatively, the user may utilize pull down menus or 
buttons to launch the optimized re-scan. However, if the 
pull down menu is used, then "save to fife", "copy to clip- 
board", and "print" are the only options available. Thus, 
the resulting image data wfll not automatically appear in 
an open application if this option is utilized. 
[0019] Both OCR (optical character recognition) pack- 
ages and the Click and Select User Interface for Docu- 
ment Scanning provide automated segmentation of text 
reports. Lines, paragraphs, and even columns of text 
are generated by these text-processing methods. How- 
ever, neither of these two solutions is convenient for 
selecting only specific portions of text within a text 
regon. such as a few fines from a single paragraph, or 



multiple lines from adjacent paragraphs, or disjoint text 
from various text regions in the preview scan of a docu- 
ment or documents. OCR packages wOl typicaDy con- 
vert an entire set of paragraphs into their ASCII form, 

5 after which the user wiD need to make corrections and 
delete the words they (fid not wish to select. Similarly, 
when using the Click and Select User Interface for Doc- 
ument Scanning, often more text than the user actually 
intended to select wfll be generated from image 

w processing the bitmapped data in the preview scan. The 
user will have to change the outlining boundary (as is 
also the case for OCR) to select only the specific text 
that they are interested in. This is a process that can be 
confusing and cumbersome if the text region cannot be 

is bounded rectangularly, as when the specific text does 
not start or end at the start and end of lines in the text 
column or paragraph, or when the text wraps around an 
image, etc. 

[00201 To enable selection of specific portions of text, 

20 the present invention employs a high-lighter function as 
part of the user interface for scanner software. The 
scanning software provides a high-lighter icon, which is 
a bitmapped tool, at the user interface level. The high- 
lighter icon can be activated to replace the normal 

25 mouse cursor either by a tool bar menu, or by right click- 
ing the mouse to access a pop-up menu, and then 
selecting an "Activate High-Lighter" option from the tool 
bar menu or the pop-up menu. When the user wants to 
highlight text the user hokte down the left mouse button 

30 and drags the active edge of the high-lighter icon over 
the desired portions of text displayed on the monitor. 
The non-text or "background" pixels on the monitor tra- 
versed by the active edge of the high-lighter icon are 
changed to a different color, such as yellow or pink, to 

35 visually indicate to the user which portions of text have 
been traversed by the high-lighter icon. The text pixels 
are left unchanged, giving the visual effect one would 
have if a real high-lighter pen were used on a real piece 
of paper containing text The user interface can be 

40 returned to the default (non-high-lighter) mode by a tool 
bar menu, or by right clicking the mouse to access a 
pop-up menu, and selecting a "Deactivate High-Lighter" 
option from the tool bar menu or the pop-up menu. 
[0021 ] After high-lighting text, in the preferred embed- 

45 iment of the invention, the image processing routine of 
the Click and Select User Interface for Document Scan- 
ning uses the set of high-lighted pixels as the substitute 
input instead of a dick point Text regions are generated 
from these pixels until the selected text exhausts all of 

so the high-lighted pixels. The regions are then cropped as 
necessary to prevent adding non-high-lighted text at the 
beginning and end of high-lighted lines. A polygonal 
bounding box is then displayed round the selected por- 
tions of text and the background pixels that were 

55 changed to a different color are restored to their original 
color. In an alternative embodiment of the invention, all 
the background pixels within the polygonal bounding 
box are changed to the high-lighter color. An optimized 
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re-scan can now be performed utffizing the bitmapped 
data represented by the selected text areas within the 
polygonal bounding boxes. 

DESCRIPTION OF THE DRAWINGS 

[0022] The above and other aspects, features, and 
advantages of the invention will be better understood by 
reading the following more particular description of the 
invention, presented in conjunction with the following 
drawings, wherein: 

FIG. 1 shows a block diagram of a computer system 
incorporating the user interface high-lighter function 
for scanner software of the present invention; 
FIG. 2 shows a representation of the screen display 
of a computer monitor shewing the high-fighter icon 
of the user interface high-lighter function for scan- 
ner software of the present invention; 
FIG. 3 shows a portion of the screen cf splay from 
FIG. 2 that has had text portions high-lighted utiliz- 
ing the high-fighter icon of the user interface high- 
fighter function for scanner software of the present 
invention; 

FIGS. 4A, 4B, and 4C show, from the portion of the 
screen display of FIG. 3, the results of image 
processing the high-lighted pixels by displaying 
polygonal bounding boxes surrounding the selected 
portions of text 

FIGS. 5A, 5B, and 5C show a block diagram of the 
overall flow of the operation of the user interface 
high-lighter function for scanner software of the 
present invention. 

FIG. 6 shows a block diagram of the un-high-lighter 
mode of the user interface high-lighter function for 
scanner software of the present invention. 

BEST MODE FOR CARRYING OUT THE INVENTION 

[0023] The following description is of the best pres- 
ently contemplated mode of carrying out the present 
invention. This description is not to betaken in a fimrting 
sense but is made merely for the purpose of describing 
the general principles of the invention. The scope of the 
invention should be determined by referencing the 
appended claims. 

[0024] FIG. 1 shows a block diagram of a computer 
system incorporating the user interface high-lighter 
function for scanner software of the present invention. 
One skilled in the art will recognize that many other con- 
figurations are possfole for accessing a scanning device 
with a computer system Only one configuration is 
shown for simplicity. Referring now to FIG. 1, computer 
system 100 contains a processing element 102. 
Processing element 102 communicates with other ele- 
ments of computer system 100 over a system bus 104. 
A keyboard 106 allows a user to input information into 
computer system 1 00 and a monitor 110 allows compu- 



ter system 100 to output information to the user. A 
graphical input device 108 is also used to input informa- 
tion. In the preferred embodiment of the invention, 
graphical input device 108 is a mouse. Scanner dance 
5 114 is also used to input information to computer sys- 
tem 1 00. Scanner device 114 may be a flatbed scanner, 
a portable hand held document scanner, or any other 
digital capturing device. 

[0025] Storage device 112 is used to store data and 

w procpams within computer system 100. Communica- 
tions interface 116. also connected to system bus 104, 
receives information from sources outside of computer 
system 100. A memory 118. also attached to system 
bus 104, contains an operating system 120, window 

75 manager 122, and scanner software 124 having the 
user interface high-fighter function for scanner software 
of the present invention. In the preferred embodiment of 
the invention, operating system 120 is the Microsoft 
Windows ® operating system. Memory 118 also con- 

20 tains first software application 1 26 and second software 
application 128. One skilled in the art will recognize that 
many more software applications could reside in mem- 
ory 118. Only two are shewn for simplicity. 
[0026] FIG. 2 shows a representation of the screen 

25 display from monitor 110 (FIG. 1) showing the high- 
lighter icon of the user interface high-lighter function for 
scanner software 124 (FIG. 1) of the present invention. 
Referring now to FIG. 2, scanner software window 200 
shows a variable resolution preview window 202 con- 

30 taining variable resolution preview scan 204 of a docu- 
ment or documerrts that have been scanned by scanner 
device 1 14 (FIG. 1). A user initiates variable resolution 
preview scan 204 by pressing a preview scan mode but- 
ton on scanner device 114, or through click input on a 

35 scan button or pull down menu presented to the user on 
monitor 110 through scanner software 124, which 
sends a signal causing scanner device 114 to use a pre- 
view scan mode to scan the document or documents in 
position to be scanned by scanner device 1 14. A pre- 

40 view scan mode is typically a low resolution scan, 
dependent upon the resolution of the display and the 
size of the preview window. A user may also initiate a 
zoom scan of an area of the document or documents 
positioned to be scanned by scanner device 114, in 

45 which case the resolution of the data contained in varia- 
ble resolution preview window 202 may be that of scan- 
ner device 114. which could be a considerably higher 
resolution than the preview scan mode. The bitmapped 
data generated from the preview scan mode is dis- 

so played as variable resolution preview scan 204 in varia- 
ble resolution preview window 202. Variable resolution 
preview scan 204 shows regions of various types, 
including text regions 206, black and white line art 
regions 208, color photograph regions 210. and gray 

55 scale photograph region 212, which is partially stperim- 
posed on one of the color photograph regions 210. 
[0027] When a user selects the "Activate High- 
Lighter" option, the normal mouse cursor is replaced by 
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a high-Oghter icon 214, which in FIG. 2 has been posi- 
tioned by the user utilizing graphical input device 108 
(FIG. 1) within predominantly text region 216. In the pre- 
ferred embodimerrt of the invention, graphical input 
device 108 fe used to drive high-Oghter icon 214. 
[0028] Variable resolution preview window 202 also 
contains pull down menu bar 218 and tool bar 220, 
which provide the user with access to various functions 
of scanner software 124. If no region has been selected, 
status bar 222 displays information regarding variable 
resolution preview scan 204 in its entirety. If a region 
has been selected , status bar 222 displays current infor- 
mation regarding the region selected. Box 224 will indi- 
cate the type of region, such as gray scale photographic 
image, color image, text, etc. Box 226 indicates the 
image file size in KB. A scaling factor is shown in box 
228. A cfifferent scaling factor may be displayed if an 
inter-application communication fink, such as TWAIN or 
OLE, has been established indicating a preferred final 
size of the regon of interest, or if the user has specified 
ona Box 230 indicates the unit of measure for a 
selected image, such as centimeters, inches, points, or 
pixels. Box 232 indicates that the height of a selected 
image, and box 234 indicates the width of a selected 
imaga Box 236 indicates the current resolution in dots 
per inch. 

[0029] Three floating windows are also shown in scan- 
ner software window 200 that present control tools to 
the user. The controls reflect adjustments made to vari- 
able resolution preview scan 204 based on the contents 
of the preview window. The user may use the tools pre- 
sented in the floating windows to further manipulate a 
selected region of interest 

[0030] Exposure adjustment window 238 offers con- 
trol tools that apply to all photographic output data types 
to preserve highlight and shadow detail. Color adjust- 
ment window 240 applies only to color photographic 
output data types to adjust hue and saturation. Black 
and white threshold window 242 applies only to black 
and white binary output data types such as One art cfip 
art, halftones, and text 

[0031] FIG. 3 shows a portion of the screen display 
from FIG. 2 that has had text portions high-lighted utitiz- 
ing the high-fighter icon of the user interface high-lighter 
function for scanner software of the present invention. 
Referring now to FIG. 3, predominantly text region 216 
is shown having four text regions 206 and color photo- 
graph region 210. A user has held down the left mouse 
button and dragged the active edge 244 of high-lighter 
icon 21 4 (FIG. 2) over three separate portions of the text 
regions 206. indicated by highlight area 302, highlight 
area 304, and highlight area 306. High-Oghter icon 214 
may be made available to the user in several different 
sizes for ease of use in selecting text of different sizes. 
Thus, the length of active edge 244 may vary such that 
a differing number of pixels may be selected by the 
active edge 244. The button down input selects a begin- 
ning line of pixels that fie along the active edge 244 at 



the button down starting location. The drag input selects 
a region of pixels traversed by the active edge 244 dur- 
ing an interim movement. The button up input at the end 
of the drag input interim movement selects an ending 

5 line of pixels that lie along the active edge 244 at the 
button up enrjng location. In the preferred embodiment 
of the invention, the background portions of the pixels 
traversed by the active edge 244 of high-lighter icon 21 4 
are cfispiayed in a different color from the current back- 

io (pound color for the background pixels as movement 
occurs. The different color, such as yellow or pink, visu- 
ally indicates to the user which portions of text have 
been traversed by the active edge 244 of high-lighter 
icon 214 as movement occurs, while allowing the text 

is pixels within the high-lighted area to still be visible in 
their original color. This is done using a simple thresh- 
olding (binarizing) algorithm to distinguish foregxxjnd 
from background. 

[0032] FIGS. 4A, 4B, and 4C show, from the portion of 

20 the screen display of FIG. 3, the results of image 
processing the high-lighted pixels by (fisplaying polygo- 
nal bounding boxes surrouncfing the selected portions 
of text Referring now to FIG. 4A, after the user has 
high-lighted the desired portions of text in FIG 3 with 

25 high-fighter icon 214 (FIG. 2), the user accesses a tool 
bar menu, or pull down menu, and selects a "Select 
Text" option. Scanner software 124 (FIG 1) receives, as 
input for an image processing routine, the high-Bghted 
pixels, which includes the beginning line of pixels, the 

30 ending line of pixels, and the pixels traversed by the 
active edge 244 (FIG. 2) of high-lighter icon 214, for 
each portion of high-lighted text The image processing 
routine begins processing with the bitmapped data rep- 
resented by the pixels within the high-Bghted areas. In 

35 the preferred embodiment of the invention, the high- 
lighted pixels serve as substitute input for the cfick point 
of the Click and Select User Interface for Document 
Scanning. Text regions are generated from the bit- 
mapped data from these high-Girted pixels unta the 

40 selected text "exhausts'' all of the high-fighted pixels. 
The regions are cropped as necessary to prevent add- 
ing non-high-lighted text at the beginning and end of 
high-fighted Ones. 

[0033] After image processing the bitmapped data 
45 within highlight area 302 (FIG. 3), the high-lighted back- 
g*ound pixels within highlight area 302 are redisplayed 
in their original background color, and a polygonal 
bounding box 416 is displayed around selected bit- 
mapped data area 402. After image processing the pix- 
50 els within highlight area 304 (FIG. 3), the high-lighted 
background pixels within highlight area 304 are redis- 
played in their original background color, and a polygo- 
nal bounding box 418 is cfisplayed around selected 
bitmapped data area 404. Cropped portions of text not 
55 selected are shown by cropped text 408 and cropped 
text 410. After image processing the pixels within high- 
light area 306 (FIG. 3), the high-lighted background pix- 
els within highlight area 306 are redisplayed in their 
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original background color, and a polygonal bounding 
box 420 is displayed around selected bitmapped data 
area 406. Cropped portions of text not selected are 
shown by cropped text 412 and cropped text 414. If the 
user now bunches an optimized re-scan, selected bit- 5 
mapped data area 402, selected bitmapped data area 
404, and selected bitmapped data area 406 will be proc- 
essed by scanner software 124. In an alternative 
embodiment of the invention, all the background pixels 
within the polygonal bounding boxes are displayed in w 
the highlight cola, and the text is cf splayed in its original 
color. 

[0034] FIG. 4B shows area 422 from FIG. 4A. Refer- 
ring now to FIG. 4B, if, after polygonal bounding boxes 
416, 418, and 420 are displayed, the user determines 75 
that too much text was automatically selected, the user 
can use high-lighter icon 214 to de-select the unwanted 
text The user accesses a tool bar menu, or pull down 
menu, and selects an "Activate Un-Highlight" option to 
activate an un-Nghlight mode. The user then holds 20 
down the left mouse button and drags the active edge 
244 of high-lighter icon 214 (FIG. 2) over the unwanted 
text The background portions of the pixels traversed by 
the active edge 244 of high-lighter icon 214 are dis- 
played in a different color from the current background 2s 
color for the background pixels as movement occurs, as 
indicated by highlight area 424, selecting a portion of 
text within selected bitmapped data area 402 bounded 
by polygonal bounding box 416. The user then 
accesses a tool bar menu, or puD down menu, and 30 
selects a "Select Text" option. The same underlying 
analysis technology is used for un-highlighting as for 
highlighting. 

[0035] FIG. 4C shows area 422 from FIG. 4B. Refer- 
ring now to FIG. 4C, scanner software 124 (FIG. 1) 35 
receives the high-lighted pixels within highlight area 424 
as input for the image processing routine, which in the 
preferred embodiment of the invention is the Cfick and 
Select User Interface for Document Scanning. The 
image processing routine processes the bitmapped 40 
data represented by the pixels within highlight area 424. 
After image processing, the high-lighted background 
pixels within highlight area 424 are redisplayed in their 
original background color, along with the any text pixels, 
as cropped text 428. Altered polygonal bounding box 45 
426 is displayed around reduced selected bitmapped 
data area 430. In an alternative embodiment of the 
invention, where the background pixels within the polyg- 
onal bounding box are dsplayed in the highlight color, 
the background pixels traversed in the un-htghlight so 
mode are redisplayed in the original background color. 
[0036] If, after polygonal bounding boxes 416, 418, 
and 420 are cfisplayed, the user determines that not all 
the desired text was automatically selected, then the 
user can use high-lighter icon 214 to highlight the adcfr- ss 
tional text adjacent to one of the polygonal bounding 
boxes. The user then accesses a tool bar menu, or pull 
down menu, and selects a "Select Text" option. Scanner 



software 124 (FIG. 1) receives the new high-lighted pix- 
els from the additional text as input for the image 
processing routtna The image processing routine proc- 
esses the bitmapped data represented by the high- 
lighted pixels. After image processing, the resulting new 
bitmapped area is added to the current bitmapped area 
to create an expanded selected bitmapped data area, 
the high-lighted background pixels within the new bit- 
mapped area are redisplayed in their original back- 
ground color, and an expanded polygonal bounding box 
is displayed around the expanded selected bitmapped 
data area (not shown in FIG. 4). 
[0037] FIGS. 5A, 5B, and 5C show a block diagram of 
the overall flow of the operation of the user interface 
high-fighter function for scanner software of the present 
invention. The user interface is called from scanner soft- 
ware 124 (FIG. 1), when requested by the user of the 
scanner software 124. Scanner software 124 performs 
other scanner functions which are not part of the inven- 
tion. 

[0038] Referring now to FIGS. 5A, 5B, and 5C, after 
entry, in step 500 a variable resolution preview scan is 
initiated for a document or documents by scanner 
device 114 (FIG. 1) that may contain bitmapped data 
representing text line art, and/or color and black and 
white photographic images of interest In step 502, var- 
iable resolution preview scan 204 (FIG. 2) representing 
the bitmapped data is displayed within variable resolu- 
tion preview window 202 (FIG. 2) in monitor 110 (FIG. 
1). 

[0039] Step 504 determines if input is received to acti- 
vate the user interface high-lighter function, such as 
through click input received from graphical input device 
108 (FIG. 1) on an "Activate High-Lighter option from a 
tool bar menu, or from a pop-up menu accessed by right 
clicking graphical input device 108. If no input to activate 
the user interface high-fighter function is received in 
step 504, control returns to scanner software 124 (FIG. 
1) where the user may access other functions within 
scanner software 124. If input to activate the user inter- 
face high-lighter function is received in step 504, then in 
step 506 the current mouse cursor is replaced with high- 
lighter icon 214 (FIG. 2). 

[0040] Step 508 determines if button down, drag, and 
up input is received, indicating that the user has 
selected a portion of text within variable resolution pre- 
view scan 204 with graphical input device 108. If no but- 
ton down, drag, and up input is received in step 508, 
control transfers to step 526. If button down, drag, and 
up input is received in step 508, then in step 510 the 
background pixels traversed by the active edge 244 of 
high-fighter icon 21 4 (FIG. 2) are displayed in a different 
color from the current background color to visually incfi- 
cate the portion of variable resolution preview scan 204 
that the active edge 244 of higrvfighter icon 214 has tra- 
versed . Text pixels traversed by active edg e 244 of high- 
lighter icon 214 are left in their original color. 
[0041] Step 512 then determines if more irput is 
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received from high-fighter icon 214 for high-lighting text, 
or if input is received to cancel text highlighting, or if 
input is received to select the text that has been high- 
lighted. If more button down, drag, and up input is 
received, incBcating that the user has selected another 5 
portion of text within variable resolution preview scan 
204, then control returns to step 510. If cancel input is 
received in step 512, such as through click input 
received from graphical input device 108 on a "Remove 
Highlight" option from a tool bar menu, or from a pop-up w 
menu accessed by right cficking graphical input device 
108. then in step 514 the background pixels that were 
displayed in a different color are redisplayed in their 
original background color. Control then passes to step 
526. If input is received to select text in step 512, such 75 
as through click input received from graphical input 
device 108 on a "Select Text" option from a tool bar 
menu, or from a pop-tp menu accessed by right clicking 
graphical input device 108, then in step 516, the pixels 
traversed by the active edge 244 of high-Bghter icon 214 20 
from step 510 are image processed. In the preferred 
embodiment of the invention, the pixels traversed, 
including the beginning fine and ending line of pixels, 
are used as substitute input for the dick point of the 
Click and Select User Interface for Document Scanning. 25 
The image processing determines a selected bit- 
mapped data area, such as selected bitmapped data 
area 402 (FIG. 4). 

[0042] In step 51 8 the background pixels traversed by 
active edge 244 of high-lighter icon 214 are displayed in 30 
their original color, and a polygonal bounding box is dis- 
played around the selected bitmapped data area deter- 
mined in step 516, such as polygonal bounding box 416 
(FIGS. 4A and 48). The text pixels are cfisptayed in their 
original color. In an alternative embodiment of the invert- 35 
tion, all the background pixels within the polygonal 
bounding box are displayed in the highlight color, and 
the text pixels are displayed in their original color. 
[0043] Step 520 determines if input is received to acti- 
vate an un-highlight mode If yes, control passes to step ao 
522, which calls FIG. 6. Upon returning from FIG. 6, 
control passes to step 524. If no input is received to acti- 
vate an un-highlight mode in step 520, then step 524 
determines if more button down, drag, and up input is 
received, incficating that the user has selected another 45 
portion of text within variable resolution preview scan 
204. ff no button down, drag, and up input is received in 
step 524, control passes to step 526. ff button down, 
drag, and up input is received in step 524, then control 
returns to step 510. so 
[0044] Step 526 determines if in^iA is received to de- 
activate the high-fighter function, such as through click 
input received from graphical input device 108 on a 
"Deactivate High-Lighter" option from a tool bar menu, 
or from a pop-up menu accessed by right cl icking graph- 55 
ical input device 108. If no input to deactivate the high- 
lighter function is received in step 526. control returns to 
step 508. ff input to deactivate the high-fighter function 



is received in step 526, then in step 528 huc^-fighter icon 
214 is replaced with the normal mouse cursor. 
[0045] Step 530 determines if input is received to per- 
form an optimized re-scan of the selected bitmapped 
data area within variable resolution preview scan 204, 
such as selected bitmapped data area 402. The input 
may be drag and drop input with graphical input device 
108 of the selected area onto the desktop, a writeable 
folder, or onto an open application, or cfick input on pull 
down menus or buttons to launch the optimized re-scan. 
If no input to perform an optimized re-scan is received in 
step 530, then control passes to step 534. If input to per- 
form an optimized re-scan is received in step 530, then 
in step 532 an optimized re-scan of the selected por- 
tions of variable resolution preview scan 204 is per- 
formed by scanner device 114 through scanner 
software 124, and the result is displayed in variable res- 
olution preview window 202 (FIG. 2). 
[0046] After performing the optimized re-scan, step 
534 determines if input is received to activate the high- 
lighter function. If input is received in step 534 to acti- 
vate the high-lighter function, control returns to step 
506. If no input is received in step 534 to activate the 
high-lighter function, control returns to scanner software 
124 where the user may access other functions within 
scanner software 124. 

[0047] FIG. 6 shows a block diagram of the un-high- 
lighter mode of the user interface high-Bghter function 
for scanner software of the present invention. Referring 
now to FIG. 6, which is called from step 522 in FIG. 5, 
step 600 determines if button down and drag input is 
received from graphical input device 108 (FIG. 1) utiliz- 
ing high-lighter icon 214 (FIG. 2). ff the answer is no, 
control passes to step 608. If the answer in step 600 is 
yes, then in step 602 the background pixels traversed by 
the active edge of high-lighter icon 214 are displayed in 
a different color from the current background color to 
visually indicate the portion of variable resolution pre- 
view scan 204 that the active edge 244 (FIG. 2) of high- 
lighter icon 214 has traversed, such as highlight area 
424 (FIG. 4B). 

[0048] In step 604, when button up input is received, 
the pixels traversed by the active edge 244 of high- 
lighter icon 214 from step 602 are image processed 
using any underlying analysis technique that can readily 
form text regions. In the preferred embodiment of the 
invention, the pixels traversed, including the beginning 
line and ending fine of pixels, are used as substitute 
input for the dick point of the Click and Select User 
Interface for Document Scanning. The image process- 
ing determines a reduced selected bitmapped data 
area, such as reduced selected bitmapped data area 
430 (FIG. 4), which is derived from selected bitmapped 
data area 402 (FIGS. 4Aand 4B). 
[0049] The bitmapped area associated with cropped 
text 428 is deleted from selected bitmapped data area 
402, yielding reduced selected bitmapped data area 
430. The background pixels traversed by active edge 
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244 of high-Ggrtter icon 214 in step 602 are redisplayed 
in their original color, and an altered polygonal bounding 
box, such as altered polygonal bounding box 426 (FIG. 
4C), is displayed around reduced selected bitmapped 
data area 430. In an alternative embodiment of the 
invention, where the background pixels within the polyg- 
onal bounding box are displayed in the highlight color, 
the background pixels traversed in the un-highlight 
mode are redisplayed in the original background color, 
and the text pixels are displayed in their original color. 
[0050] Step 606 determines if more button down and 
drag input is received, ff yes, control returns to step 602. 
If no more button down and drag input is received in 
step 606, then step 608 determines if input is received 
to de-activate the un-highlight mode. If no, then control 
returns to step 600. If input is received to de-activate the 
un-highlight mode in step 608, then control returns to 
step 524 of FIG. 5B. 

[0051 ] Having described a presently preferred embod- 
iment of the present invention, it will be understood by 
those skilled in the art that many changes in construc- 
tion and circuitry and widely differing embodiments and 
applications of the invention will suggest themselves 
without departing from the scope of the present inven- 
tion, as defined in the claims. The disclosures and the 
description herein are intended to be illustrative and are 
not in any sense Rmrting of the invention, defined in 
scope by the following claims. 

Claims 

1. A user interface high-lighter method, within a com- 
puter system (100), for selecting at least one por- 
tion of text comprising the steps of: 

(a) displaying (502), on a monitor (110) con- 
nected to said computer system (100), a 
scanned image containing at least one region 
of text; 

(b) receiving (504) a first input in said computer 
system (100) to activate said user interface 
high-lighter method; 

(c) displaying (506), in said monitor (110), a 
high-lighter icon (21 4) in place of a cursor for a 
graphical input device (108), wherein said 
graphical input device (108) is connected to 
said computer system (100); 

(d) receiving (508) a first button dwn, button 
drag, and button up input from said graphical 
input device (108), wherein said high-tighter 
icon (214), further comprising an active edge 
(244), is moved by said graphical input device 
(108) within said at least one region of text, and 
further wherein a first plurality of pixels are tra- 
versed by said active edge (244) of said high- 
lighter icon (214) from said first button down, 
button drag, and button up input associated 
with said moving by said graphical input device 



(108); 

(e) receiving (512) a first select input to select 
said first plurafity of pixels as a first data input 
for an image processing routine; and 

5 (f) processing (516), by said image processing 

routine, a first portion of bitmapped data repre- 
sented by said first plurality of pixels to select 
said at least one portion of text from said at 
least one region of text within said scanned 

w image. 

2. A user interface high-lighter method, within a com- 
puter system (100), for selecting at least one por- 
tion of text according to claim 1 wherein step (a) 

is further comprises step (aO) performed before step 
(a): 

(aO) scanning (500) at least one document with 
a scanner device (114) connected to said com- 

20 puter system (100), wherein said scanning of 

said at least one document produces said 
scanned image containing said at least one 
region of text displayed on said monitor (110), 
and further wherein said scanned image is a 

25 variable resolution preview scan (204) of said 

at least one document 

3. A user interface high-Gghter method, within a com- 
puter system (100), for selecting at least one por- 

30 loon of text according to claim 1 wherein step (d) 
further comprises step (d1): 

(d1) cfisplaying (510) in said monitor (110) a 
first plurality of background pixels within said 
35 first plurality of pixels in a different color from a 

current background color for said first plurality 
of background pixels. 

4. A user interface high-Gghter method, within a com- 
40 puter system (100), for selecting at least one por- 
tion of text according to claim 3 further comprising 
the steps of: 

(g) displaying (518) said first plurality of back- 
45 ground pixels within said first plurality of pixels 

in said current background color; and 

(h) displaying (518) a polygonal bounding box 
(416, 418, 420) around said at least one por- 
tion of text 

50 

5. A user interface high-Gghter method, within a com- 
puter system (100). for selecting at least one por- 
tion of text according to claim 4 further comprising 
the steps of: 

55 

(i) receiving (520) a second input in said com- 
puter system (100) to activate an un-higfilight 
mode of said user interface high-lighter 
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method; 

(j) receiving (600) a second button down, but- 
ton drag, and button up input from moving said 
graphical input device (108) within said at least 
one portion of text, wherein a second plurality s 
of pixels are traversed by said active edge 
(244) of said high-fighter icon (214) from said 
second button down, button drag, and button 
up input associated with said moving of said 
graphical input device (108); 10 
(k) displaying (602) in said monitor (1 1 0) a sec- 
ond plurality of background pixels within said 
second plurality of pixels in said different color 
from said current background color for said 
second plurality of background pixels; is 
(I) receiving (604) said second plurality of pix- 
els as a second data input for said image 
processing routine; 

(m) processing (604). by said image process- 
ing routine, a second portion of bitmapped data 20 
represented by said second plurality of pixels to 
select a portion of said at least one portion of 
text; 

(n) deleting (604) said second portion of bit- 
mapped data from said first portion of bit- 2s 
mapped data to form a reduced selected 
bitmapped data area (430). wherein said at 
least one portion of text is reduced to form a 
reduced said at least one portion of text from 
said at least one region of text within said 30 
scanned image, and further wherein said 
reduced said at least one portion of text is rep- 
resented by said reduced selected bitmapped 
data area (430); and 

(0) displaying (604) an altered polygonal 35 
bounding box (426) around said reduced said 

at least one portion of text. 

6. A user interface high-lighter method, within a com- 
puter system (100), for selecting at least one por- 40 
tion of text according to claim 3 further comprising 
the steps of: 

(g) displaying (518) a polygonal bounding box 
(416, 418, 420) around said at least one por- 45 
tion of text; 

(h) receiving (520) a second input in said com- 
puter system (100) to activate an un-highlight 
mode of said user interface highlighter 
method; so 

(1) receiving (600) a second button down, but- 
ton drag, and button up input from moving said 
graphical input device (108) within said at least 
one portion of text, wherein a second plurality 

of pixels are traversed by said active edge ss 
(244) of said high-lighter icon (214) from said 
second button down, button drag, and button 
up input associated with said moving of said 



graphical input device (108); 
(D displaying (602) in said monitor (1 1 0) a sec- 
ond plurality of background pixels within said 
second plurality of pixels in said current back- 
ground color. 

(k) receiving (604) said second plurality of pix- 
els as a second data input for said image 
processing routine; 

(I) processing (604). by said image processing 
routine, a second portion of bitmapped data 
represented by said second plurality of pixels to 
select a portion of said at least one portion of 
text; 

(m) deleting (604) said second portion of bit- 
mapped data from said first portion of bit- 
mapped data to form a reduced selected 
bitmapped data area (430), wherein said at 
least one portion of text is reduced to form a 
reduced said at least one portion of text from 
said at least one region of text within said 
scanned image, and further wherein said 
reduced said at least one portion of text is rep- 
resented by said reduced selected bitmapped 
data area (430); and 

(n) displaying (604) an altered polygonal 
bounding box (426) around said reduced said 
at feast one portion of text 

7. A user interface high-lighter method, within a com- 
puter system (100), for selecting at least one por- 
tion of text according to claim 1 wherein said first 
button down, button drag, and button up input fur- 
ther comprises a first button down input a first but- 
ton drag input and a first button up input, and step 
(d) further comprises steps (d1) through (d3): 

(d1) identifying, from a first starting location on 
said monitor (110) with said active edge (244) 
of said high-lighter icon (214) from said first 
button down input a first beginning line of at 
least two pixels of said plurality of pixels; 
(d2) identifying, from a first interim movement 
on said monitor (110) with said active edge 
(244) of said high-lighter icon (214) from said 
first button drag input, a first interim region of 
pixels of said plurafity of pixels; and 
(d3) identifying, from a first ending location on 
said monitor (1 10) with said active edge (244) 
of said high-lighter icon (214) from said first 
button up input, a first ending line of at least two 
pixels of said plurality of pixels. 

& A user interface high-lighter method, within a com- 
puter system (100), for selecting at least one por- 
tion of text according to claim 1 further comprising 
the steps of: 

(g) repeating steps (d) and (e) for receiving a 
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second button down, button drag, and button 
up input from said graphical input device (108), 
wherein a second plurality of pixels are tra- 
versed by said active edge (244) of said high- 
lighter icon (214), and receiving a second s 
select input to select said second plurality of 
pixels as a second data input for said image 
processing routine; and 
(h) repeating step (f). wherein said processing 
by said image processing routine of a second w 
portion of bitmapped data represented by said 
second plurality of pixels selects a next portion 
of text from said at least one region of text 
within said scanned image. 

15 

9. A user interface high-lighter method, within a com- 
puter system (100), for selecting at least one por- 
tion of text according to claim 1 further comprising 
the steps of: 

20 

(g) receiving (530) a second input in said com- 
puter system (100) to perform an optimized re- 
scan of said at least one portion of text from 
said at least one region of text within said 
scanned image; 25 

(h) performing (532), with scanner software 
loaded into a memory (118) in said computer 
system (100), said optimized re-scan of said at 
least one portion of text from said at least one 
region of text within said scanned image; and 30 

(i) displaying (532) a result of said optimized re- 
scan in said monitor (110). 

10. A user interface high-lighter method, within a com- 
puter system (100), for selecting at least one por- 35 
tion of text according to claim 1 wherein said 
graphical input device (108) is a mouse. 
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